Global Convergence of Policy Gradient Primal–Dual Methods for Risk-Constrained LQRs

While the techniques in optimal control theory are often model-based, policy optimization (PO) approach directly optimizes performance metric of interest. Even though it has been an essential for reinforcement learning problems, there is little theoretical understanding its performance. In this article, we focus on risk-constrained linear quadratic regulator problem via PO approach, which requires addressing a challenging nonconvex constrained problem. To solve it, first build our earlier result that time-invariant affine structure to show associated Lagrangian function coercive, locally gradient dominated, and local Lipschitz continuous gradient, based establish strong duality. Then, design primal–dual methods with global convergence guarantees both model-based sample-based settings. Finally, use samples system trajectories simulations validate methods.